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UPPER  BOUNDED  VARIABLES  IN  LINEAR  PROGRAMMING 
G.  B.  Dantzig  and  S.  M.  Johnson 


1 .  INTRODUCTION 

With  the  growing  awareness  of  the  potentialities  of  the 
linear  programming  approach  to  both  dynamic  and  static 
problems  of  industry,  of  the  economy,  and  of  the  military, 
the  main  obstacle  toward  full  application  is  the  inability  of 
current  computational  methods  to  cope  with  the  magnitude  of 
the  technological  matrices  for  many  common  situations. 
However,  in  certain  cases,  such  as  the  now  classical 
Hitchcock-Koopmans  transportation  model,  it  has  been  possible 
to  solve  the  linear  Inequality  system  in  spite  of  size 
because  of  simple  properties  of  the  system.  This  suggests 
that  considerable  research  be  undertaken  to  exploit  certain 
special  matrix  structures  in  order  to  facilitate  ready 
solution  of  larger  systems. 

Indeed,  recent  computational  experience  has  made  it 
clear  that  standard  techniques  such  as  the  simplex  algorithm, 
which  have  been  used  to  solve  successfully  general  systems 
involving  two  hundred  equations  on  electronic  computers 
(in  any  reasonable  number  of  non-negative  unknowns),  are  too 
^tedlQUB^and  lengthy  ^to  be  practical  for  extensions  much 
beyond  this  figure,  -wr  purpon?  h4r?  "develop 

short-cut  computational  methods  for  solving  an  important 
class  of  systems  involving  upper  bound  restraints  on  the 
variables 


\ 
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Upper  bounds  on  variables  In  linear  programming  are 
used  quite  commonly.  Their  use  tends  to  Increase  as  models 
become  more  realistic.  Typically,  in  practice,  this  would 
mean  that  more  activities  enter  the  solution  since  the 
favored  activities  are  not  allowed  to  carry  the  whole  load. 

The  following  two  classes  of  problems  illustrate  the 
use  of  upper  bounds. 

A.  Capacity  Restrictions 

1.  On  factory  production  rates. 

2.  On  traffic  links  in  a  transportation  problem. 

B.  Convex  Programming  Problems 

1.  Many  examples  where  the  cost  of  an  activity  is 
a  convex  function  of  the  level  engaged  in. 

2.  Linear  programming  under  uncertainty. 

2.  The  Statement  of  the  Problem. 


Consider  a  system  of  equations 

(i)  k*"*3  m  bi  Xj  -  °  (i  ■ i#  2> 


-  Z, 


where  it  is  desired  to  obtain  values  of  such  that  the  form 
£  cjxj  *8  to  be  minimized. 

The  size  of  the  array  of  coefficients  associated  with  awh 
a  linear  programming  problem  may  become  uncomfortably  large 
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when,  In  addition  to  (1),  many  (or  all)  variables  of  the 
Initial  set  have  upper  bounds.  Thus,  if  each  variable 
satisfied  o  <  x4  <  aJ#  it  is  customary  to  add  an  additional 
variable,  say  x^,  and  a  new  equation 

(2)  *j  +  X*  -  aj  (*j  >  0.  x.  >  0) 

to  take  care  of  each  such  restriction. 

We  shall  refer  to  the  original  system  plus  these  upper 
bound  restrictions  as  the  enlarged  system. 

To  illustrate,  a  linear  programming  problem  of  the 
transportation  type  involving  m  destinations  and  n  origins 
has  a  matrix  involving  m  ♦  n  rows  and  m  •  n  variables  x^ 
associated  with  m  •  n  possible  routes  Joining  origins  with 
destinations.  Suppose  there  is  a  capacity  limitation  r^  on 
a  route  so  that,  in  addition  to  the  original  system  of 
equations  and  linear  inequalities,  0  <  x^,  one  must  Impose 
m  *  n  additional  restraints 

XU  +  xij  “  rij  (XU  ^  °»  xij  £  °)* 

It  is  clear  now  that  the  original  system  has  been  expanded  to 
(mn  +  m  +  n)  rows  and  2m  •  n  variables.  A  transportation 
problem  with  these  added  restraints  is  also  called  a 
"Capacitated  Hitchcock  Problem"  [4] . 

3«  The  Technique  Illustrated: 

The  main  idea  which  permits  solutions  to  enlarged 
systems  without  very  much  more  effort  than  smaller  systems  is 
based  on  a  slight  generalization  of  the  simplex  procedure. 
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The  simplex  algorithm  is  usually  applied  with  the  non-baslc 
or  Independent  variables  at  the  fixed  value  zero.  However, 
a  little  reflection  makes  It  clear  that  the  Independent 
variables  could  be  at  any  fixed  value*  The  simplex  criterion 
Indicates  that  it  pays  to  Increase  the  value  of  a  variable 
xt  (and  adjust  the  values  of  basic  variables)  If  c  <  o. 

D  8 

However,  If  x_  Is  at  Its  upper  bound  value  this  will  not  be 

B 

possible.  Similarly  if  cl  >  0  It  pays  to  decrease  the  value 
of  x  unless  x  is  at  its  lower  bound  value  (usually  zero). 

B  B 

If  xg  is  neither  at  its  upper  nor  lower  bound  value  It  pays 

to  Increase  or  decrease  its  value  according  as  c_  <  0  or 

8 

C,  >  0. 

The  purpose  of  this  section  is  to  show  that  the  upper 
bound  restraints  may  be  provided  for  by  applying  the  simplex 
algorithm  to  the  original  system  with  due  care  that  the  range 
of  values  of  a  variable  appearing  In  a  basic  solution  stays 
within  Its  upper  and  lower  bounds.  A  simple  example  will 
Illustrate  the  technique.  (The  proof  In  general  Is  easy  and 
Is  left  to  the  reader. ) 

Find  numbers 

(3)  0  £  Xj  £  4  ,  01x^5,  0  £  x3  <  1  , 

0  £  *4  £2  .  0$x5iJ 

and  minimum  z  satisfying 

The  relative  cost  coefficients,  Cj,  are  the  coefficients 
in  the  objective  form  after  elimination  of  the  basic  variables. 
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X1  +  x3 

-  2x^ 

-  3 

(4)  x2  -  Xj 

♦  *4  +  2x5 

-  4 

—  2x 

-  +  *5 

-  z  -|0. 

Using  x^  and  as  basic 

variables,  the 

basic  feasible 

solution  is 

(5)  [  3]  ,  [4]  ,  0 

,  0  ,  0 

,  0  , 

where  the  values  of  the  basic 

variables  are 

shown  by 

brackets  [  ].  It  will  be  noted  that  the  values  of  all 
variables  are  less  than  their  upper  bound  restraints.  Since 
c-j  -  -2  <  0  it  pays  to  Increase  the  value  of  the  variable  Xy 
Holding  the  values  of  the  other  non-basic  variables  fixed, 
the  values  of  the  variables  become 

(6)  [3-X3]  »  [*+*5]  »  Xy  0  ,  0  ,  -2*^  . 

However,  in  this  case  it  is  not  possible  to  Increase  x^  to 
the  value  x^  ■  3  because  0  <  x^  <  1  .  Accordingly  we 
Increase  x^  to  its  upper  bound  value  and  hold  it  fixed  there 
while  keeping  the  same  set  of  basic  variables.  The  new 
"basic  feasible  solution"  is 

(?)  [2]  ,  [5]  ,  1  ,  0  ,  0  ,  -2  . 

Since  c^  -  — 1,  we  now  proceed  to  Increase  x^,  and  the  values 


of  the  variables  become 


P-921 

4-24-57 

-6- 


(8)  [2+2xj|]  ,  [5~2Xij]  $  1  #  x^  #  0  »  “2“x4* 

The  largest  value  of  Is  not  5/2  because  0  <  x^  <  2,  nor 
Is  It  x^  •  2  ,  because  In  this  case  x^  -  2  +  2*2  -  S,  which 
violates  0  <  x^  <  4.  Indeed  the  largest  value  of  x^  Is 
x^  >  1  at  which  value  the  basic  variable  x^  hits  Its  upper 
bound  value  4.  Accordingly  we  drop  x^  from  the  basic  set, 
and  replace  It  by  x^.  The  new  canonical  form  relative  to  the 
variables  xg  and  x^  Is  obtained  by  using  -2x^  as  pivot 
element  In  (4),  thereby  obtaining 

-  \  X1  ~  ?  x3  +  x4 

(9)  \  x^  +  x2  -  \  x^  +  2x^ 

~?xl  ~ix3  +  x5 

and  the  new  basic  feasible  solution  Is 

(10)  %  ,  [3]  ,  1  ,  (ij  ,0,-3, 

since  the  variable  x^  enters  the  non-baslc  set  at  Its  upper 
bound  value  x^  ■  4.  It  will  be  noted  that  the  basic  vari¬ 
ables  are  between  their  lower  and  upper  bounds  and  the  non- 
basle  variables  are  either  at  their  lower  or  upper  bounds; 
the  one  at  Its  lower  bound,  x^,  has  Its  corresponding 

1  £  0*  while  the  others  at  their  upper  bound  values 
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x^  and  Xy  have  c ^  -  1/2  <  0,  and  c ^  —  5/2  <  0.  Slnee 

this,  as  we  shall  show  In  a  moment,  la  the  criterion  for 
optimality  In  the  bounded  variable  problem,  no  further 
Iterations  are  required. 

To  prove  optimality  In  this  case  let  us  substitute  In 
(9)  for  the  non-baslc  variables  at  upper  bounds  the 
expressions 

(11)  X1  -  *  -*i  .  *3-l  -  *3  » 

we  obtain  immediately  an  equivalent  linear  programming  problem 

2*  +  g-  x^  +  x^  -1 

(12)  -  g-  x£  +  Xg  +  g-  x^  +  2X5  "  ^ 

+  |xj  +ix3  +  *5  —  *  -  3 

where 

0  £  x*  <  4,  0  <  x2  <  5,  0  «  x  •  <  1  , 

(13) 

0  <  x4  <  2»  0  <  x5  <  3. 

It  will  be  noted  that  Xg  -  3#  x^  ■  1,  and  all  other  Xj  or 
x^  ■  0  constitute  an  optimal  basic  feasible  solution  to  (12) 
that  does  not  violate  (13)#  —  hence  must  be  an  optimal 
solution  to  (12)  and  (13).  This  solution  is  obviously  the 
same  as  (10). 
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».  C0MVKX-8BPAKAHLB  OBJECTIVE  FPMCTIOH  fe] .  b3 

Consider  a  situation  in  which  the  objective  function. 
Instead  of  being  a  linear  form  to  be  minimized,  is  of  the  form 


<») 


AW  - 


min 


(*j  £  0) 


where  i®  11  convex  funotion  and  the  Xj  are  subject  to 


(15) 


(l  1,  2,  i»i,  ci )  • 


The  trick  here  is  to  represent  ^j(x)  as  an  indefinite 
Integral  which  is  approximated  by  a  sum  over  k  fixed  intervals. 

To  see  this  clearly,  let  us  note  first  that  we  may 
represent  ^(x)  as 

*(*)  -4  *'  (u)du+  k,  ^(0)  ■  k. 


where  ^(x)  convex  means  (u)  is  nondecreasing  and  that  4%  (u) 
may  be  approximated  by  a  histogram  over  some  Interval 
0  <  u  <  c  where  c  is  some  assumed  very  large  upper  bound  for 
x*  This  is  illustrated  below. 


hl  ^  h2  1  h3  •  •  •  i  \ 
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Here  height  of  bars  h^  <  h^^  follows  from  the  convexity 
of  4*  We  now  replace  x  by 


X  -  \  +  Ag  +  . . .  +  A.n 


\  >  0 


where  are  non-negative  variables  and  k  Is  the  fixed  number 
of  Intervals  Into  which  the  range  of  x  was  subdivided.  It  Is 
easy  to  see  that 

k 

jf(x)  -  min  7  0  <  \  <  di  • 

Indeed,  It  Is  clear  that  the  minimum  Is  attained  by  choosing 
-  d1#  Ag  .  d2,  ...  until  the  value  of  x  Is  exceeded  for 
some  1  In  which  case  A^  is  set  equal  to  the  residual.  Thus 
the  effect  of  the  minimization  Is  to  represent  ^(x)  by  the 
area  under  the  histogram  up  to  x. 

Ve  now  employ  this  approach  to  solve  (14)  and  (15) •  The 
procedure  la  to  replace  Xj  and  4j  In  the  linear  programming 
problem  by 

<«>  -4  ‘I  41J 


J  ■  1#  2,  • . . i  n 


and 


(17) 


'W  m&  hiAj 


°<Au<dij  * 


noting  that  since  a  minimum  for  £  4 j (x j )  Bought  In  (14), 

this  Implies  that  the  values  of  A  satisfying  (16)  must,  at 
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the  minimum. 


satisfy  ^  hijAij 


min* 


This  manner  of  treating  convex  separable  objective 
functions  greatly  Increases  the  number  of  variables  without 
Increasing  the  number  of  equations  (Instead  of  x's  we  have 
many  more  A's,  each  with  an  upper  bound).  However,  It  Is  the 
number  of  equations  that,  as  a  rule,  determine  the  amount  of 
work  In  the  simplex  method.  Moreover,  It  should  be  noted 
that  there  are  numerous  short  cuts  possible  due  to  the 
appearance  of  soveral  columns  with  Identical  coefficients 
(except  for  the  cost  row)  so  that.  In  fact.  It  Is  quite 
simple  to  solve  rapidly  cases  Involving  a  convex  separable 
objective  form. 

5.  LINEAR  PKOQRAWmO  PMDBR  UNCERTAINTY  fe] 

An  example  Is  a  two-stage  programming  problem  with  the 
following  structure.  For  the  first  stage 


n 


where  x^j  represents  the  amount  of  the  1th  resource  assigned 
to  the  j**1  destination  and  bjj  represents  the  number  of  units 
of  demand  at  destination  J  that  can  be  satisfied  by  one  unit 
of  resource  1.  For  the  second  stage 
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(19)  **J  "  UJ  *  VJ  —  ®j  ^  •••* 

where  Vj  1b  the  shortage  of  supply  and  s^  Is  the  excess  of 
supply. 

The  total  cost  function  Is  assumed  to  be  of  the  form 


c 


l.e.#  depends  linearly  on  the  choice  of  x^j  and  on  the 
shortages  (which  depend  H>n  assignments  u^  and  the  demands 

dj). 

Our  objective  will  be  to  minimize  total  expected  costs. 
Let  ^j(ujldj)  be  bhe  cost  at  a  destination  if  the  supply  Is 
Uj  and  the  demand  la  d^.  It  Is  clear  that 


0 

v 


lf  dj  *  uj 

If  dj  OUj 


we  shall  now  give  a  result  due  to  H.  Scarf. 

Theorem:  The  expected  value  of  denoted  by 


ffj(uj).  Is  a  convex  function  of  u^. 
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Proofs  Let  p(<*j)  be  the  probability  density  of  d^;  then 


+  oo 


(x-Uj)p(x)dx 


+  00  +00 

■  a.  /  xp(x)dx  -  a.u.  /  p(x)dx 

J  X-Uj  J  J  X-Uj 


whence  differentiating  ^(u) 

oo 

^•(u.)  -  -  a  /  p(x)dx  . 

J  J  J  X»Uj 

It  is  clear  that  ^J(u^)  is  a  nondecreasing  function  of  u 
with  ^j(uj)  ^  0  and  that  ^j(uj)  1®  convex. 

Thus  the  expected  value  of  the  objective  function  is 


(20) 


Exp  c 


£  cijxu  +J£la</,j('V 


where  ^j(uj)  are  convex  functions.  Thus  the  original  problem 
has  been  reduced  to  minimizing  (20)  subject  to  (18).  This 
permits  application  of  the  device  already  discussed  for 
approximating  such  a  problem  by  a  standard  linear  programming 
problem  in  case  the  objective  function  can  be  represented  by 
a  sum  of  convex  functions. 
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